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Time scale separation and heterogeneous off-equilibrium dynamics in spin models over 

random graphs 

A. BarratQ and R. ZecchinaB 

Abdus Salam International Center for Theoretical Physics, Strada Costiera 11, P.O. Box 563, 34100 Trieste (Italy) 

We study analytically and numerically the statics and the off-equilibrium dynamics of spin models 
over finitely connected random graphs. We identify a threshold value for the connectivity beyond 
which the loop structure of the graph becomes thermodynamically relevant. Glauber dynamics 
simulations show that this loop structure is responsible for the onset of dynamical features of a 
local character (dynamical heterogeneities and spontaneous time scale separation) , consistently with 
previous (experimental and numerical) studies of glasses and spin glasses in their approach to the 
low temperature phase. 



Among the characteristic features of the equilibrium 
and off-equilibrium behavior of complex physical systems 
such as glasses or granular materials, two basic ones are 
the heterogeneities occurring in the spatial distribution of 
particles and the time scale separation in the relaxation 
processes of the different degrees of freedom The 
issue of building a clear connection between such local 
structures and the global relaxation process is currently 
an open basic topic. In this context, simple spin models 
have played a crucial role, both for the analytical studies 
which have provided a possible mean-field theory of the 
glass transition || and for numerical simulations. In par- 
ticular, the study of mean field spin-glasses with infinite 
connectivity has allowed to understand some features of 
both the statics || and dynamics of the glassy phase . 
However, intrinsic to the above models is the topology 
of the connections which cannot account for any local 
structure, each site being connected with all others. 

In this Letter, we will discuss the relationship between 
the topology of interactions, in the spirit of random net- 
works (|||, and the onset of heterogeneous glassy dy- 
namics in models which allow some analytical under- 
standing, namely interacting spin models defined over 
random graphs with finite connectivity. We will provide 
analytical and numerical results which clarify some basic 
differences between infinitely connected and finitely con- 
nected mean field spin models and, more interestingly, 
identify a simple link between the loop structure of the 
random lattice, the nature of the couplings and the glassy 
heterogeneous dynamics. As we shall discuss, in finitely 
connected mean field models the non-trivial underlying 
topological structure of the links is responsible for the ap- 
pearance of non-equilibrium phenomena of local nature 
(both in space and time) and the numerical results turn 
out to be in remarkable agreement with those of finite 
dimensional systems. Interestingly enough, our results 
appear also to be related with the so called "small- world" 
dynamical patterns discussed in ref. JlQ] , of relevance in 
contexts different from physics, e.g. biological oscillators, 
neural networks, spatial games and genetic control net- 
works: all these problems are indeed defined over random 



graphs where the typical separation between two vertices 
in the graph is much lower than in regular graphs, allow- 
ing for a quick spread out of dynamical correlations over 
the network (see JhJ and references therein for a more 
detailed discussion). 

Given a graph Q = (V,E), where V is the set of N 
vertices and E is the set of bonds joining K = 2 (graphs) 
or K > 2 (hyper-graphs) vertices, the spin Hamiltonians 
take the form 



H = 



E 

ii<...<i K eV 



Ji, 



(1) 



where the indices i%, ...,ik run over the set V of N ver- 
tices, each vertex i bearing an Ising spin Si, and the cou- 
plings Ji 1 .... t i K associated to the random bonds assume 
values of order one (to be compared with 0(1/ V^V) as in 
usual infinitely connected models). We consider graphs 
with finite connectivity, in which the notion of distance 
is simply the minimal number of bonds on a path con- 
necting two sites, also referred to as chemical distance. 

In the study of random (hyper)graphs the control pa- 
rameter is the average density of bonds, 7 (or the aver- 
age connectivity C — Klj). For densities small enough, 
the graph consists of many small connected clusters of 
size 0(lnN). If 7 increases up to the percolation value 
7c there appears a spanning cluster containing a finite 
fraction of the N sites in the limit of large N. How- 
ever, such a spanning cluster can a priori have a tree- 
like structure, for which the randomness of the couplings 
Jix,...,i K = ±1 can be eliminated by a gauge transforma- 
tion on the spins (just like in the one-dimensional random 
bonds Ising model) . This leads to the definition of a sec- 
ond threshold value for the density 7, defined as that 
critical density 70 at and beyond which frustration in 
the system cannot be removed by such a gauge transfor- 
mation, and therefore gives a macroscopic contribution 
to the thermodynamics, raising both internal energy and 
entropy in the system. Geometrically this threshold cor- 
responds to the appearance of an extensive number of 
loops in the spanning cluster and it has been named per- 
colation of order (PO) transition in ref. plj|. While, for 
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random K = 2 graphs, the two transitions are known to 
coincide here we shall study the K — 3 hypergraph 
structure. While the notion of loops in hypergraphs is 
rather counter intuitive, the idea of frustration retains 
its simple physical interpretation. The study of the on- 
set of frustration in the ground state phase diagram of 
the associated random K — 3 spin glass model, will allow 
us to show that the percolation transition j c and the PO 
transition 70 are well separated. Interestingly enough, 
by resorting to extensive Glauber dynamics simulations, 
we shall also show that such a change in the graph struc- 
ture is responsible for the onset of heterogeneous glassy 
dynamics. 

the most striking geomet- 



As discussed in refs. |12 



rical feature characterizing the ground state phase dia- 
gram of frustrated spin models over finite connectivity 
random graphs above the PO transition, is that, in spite 
of a finite entropy per site, there exists a finite fraction 
of spins which is totally constrained, a "backbone" that 
does not change from state to state (strongly reminis- 
cent of rigidity percolation ) . The remaining fraction 
of spins is weakly constrained and accounts for the overall 
exponential degeneracy of the ground state. 

Here we are interested in the dynamical, off- 
equilibrium, consequences of such a structure consisting, 
as we shall see, in a spontaneous separation of weakly 
constrained spins (i.e. dynamically fast) and strongly 
constrained ones, leading to time scale separation and 
heterogeneous dynamics at sufficiently low temperatures. 
Such a behavior, by definition typical of glassy systems 
turns out, for K > 3, to be independent on the frus- 
tration of the couplings in that the underlying loop struc- 
ture together with the K-body interaction lead to an an- 
nealed self-induced "geometrical" frustration. 

K = 3 hypergraphs are constructed as follows: given 
N sites, we choose at random triplets (i,j,k) for 
which the couplings Jijk will be non zero. For 7 greater 
than the percolation value 7 C = 1/3 (7 C = k(k-i) 
K-spin couplings), we obtain, as previously explained, a 
spanning cluster of connected sites, containing a finite 
fraction of the N spins in the large N limit, and many 
other smaller (order log N) disconnected clusters []l6| . 

The PO transition is simply identified by comparing 
the ground state energy of the random J^k — ±1 system 
with that of the ferromagnetic system defined over the 
same hypergraphs. The value of 7 beyond which the two 
energies start to deviate identifies 70. Such a calculation 
allows to identify a wide gauge region, j c < 7 < 70, 
where the relevant structure of the spanning cluster is 
tree-like. 

Within the so called Replica Symmetric (RS) func- 
tional framework of diluted spin glasses jL{|, we find a 
first order PO transition at Jq S ~ 0.9, characterized 
by a finite backbone and finite entropy at the thresh- 
old. Such a result was derived by adopting the full RS 



iterative scheme discussed in ref. |T^] in the resolution of 
the self-consistency equation for the T = probability 
distribution of the effective local fields P(h) = 2R(h/2) 
(in the notation of ref. which is given by R(y) = 

If cos (^/)exp {-97(1 x])} wher e <f>[R;x] = 

J n ^ o dz\dz<2,R{z\)R{zi) cos[xmin(l, \z\\, \z2\)\- Looking 

for solutions of the form R(y) = YleL-oc r Z & \ V ~ f ) > 
where 1 jp is the resolution of the field which eventually 
goes to zero, one obtains a set of p + 1 coupled equa- 
tions in the independent variables {r>} (£ = 0, l,...,p). 
Once this set has been solved, the ground state energy 
can be easily derived and compared with that of the cor- 
responding ferromagnetic model, which, as expected, is 
simply proportional to the average connectivity and cor- 
responds to the trivial re = (I 7^ 0) and r*o = 1 RS 
solution. While the exact identification of the threshold 
value 70 would require a full Replica Symmetry Break- 
ing solution, indeed an open problem, the qualitative fea- 
tures of the phase transition are correctly identified al- 
ready at the RS level jl8). In order to check this fact, we 
have done exhaustive enumeration of finite systems with 
sizes N = 18, 20, ...28 (averaged over 18000, 15000, ...3000 
samples respectively). The extrapolated value for the 
threshold is 70 — 0.8 — 0.9 (with a value of the backbone 
at the transition of ~ 0.2), slightly below 7 ( ^ s ' (~ .9 for 
p = 15) as expected. However, both the nature of the 
phase transition and the dependence of the ground state 
energy on 7 for large connectivity are consistent with the 
RS solution. 

For brevity, here we do not report explicitly the details 
of the above analysis (a similar calculation is thoroughly 
described in ref. |12|] ) , but rather focus on the dynamical 
consequences of the topological structure arising from the 
ground state analysis. Results from extended Glauber 
dynamics simulations performed over various graphs and 
Hamiltonians, indeed, confirm the appearance of the ex- 
pected time scale separation and heterogeneous structure 
of the dynamics in the low temperature phase, where the 
systems are out of equilibrium at all times, and display 
aging dynamics. The topology of the connections rather 
than the form of the interactions appear to be the source 
of the robustness of the phenomenon. Such a feature is 
well known in models of glassy material 

The basic tool used to detect aging dynamics is 
the spin-spin correlation function, C'(t w + t,t w ) — 
■h X^j=i ( s i{tw + t)si(t w )), where t w is the so called wait- 
ing time and the average is taken over the spins, the 
thermal noise ((.)) and the disorder (over-line): while, 
for equilibrium dynamics, C(t w +t,t w ) depends only on 
t, aging is defined by the fact that C(t w + i, t w ) depends 
also on t w for all times. To look for heterogeneities in a 
system, we have instead to study more local quantities, 
like local correlations Ci(t w + t,t w ) = (si(t w + t)si(t w )) 
(averaged over the initial conditions), and the individual 
rates of flipping of the spins. For each spin, we can reg- 
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ister during one Monte Carlo run the number of times 
it flips, and deduce the mean time r, between two spin- 
flips. Of course, the total running-time T rnax gives an 
upper cut-off. We can then look at the distribution of , 
averaged over the samples, P(t) p7[ . 

We have simulated graphs with either fixed connec- 
tivity (4 and 6) or fluctuating connectivity (values of C 
ranging from 3 to 18), and either ferromagnetic or ran- 
dom (±1) couplings. To compute P(t), we used sizes 
from 500 to 5000 spins and a number of samples varying 
from 50 to 100. No relevant finite size effect was ob- 
served, consistently with the self-averaging character of 
P(t). The dynamics has been implemented as a Glauber 
algorithm with random updating of the spins, with the 
runs mostly performed up to a time of 10 6 Monte Carlo 
steps per spin. For consistency, we have also looked at 
times up to 10 7 MC steps per spin for some samples. 

At high temperature, we find of course a quite sim- 
ple P(t), peaked at small values of r, i.e. high flipping 
rates, for all connectivities. However, as the tempera- 
ture is lowered, we observe very different behaviors for 
different mean connectivities. Let us first concentrate on 
the case of fixed connectivity. For a random hyper-graph 
with connectivity 4, P(t) is a smooth function peaked 
around a mean value (increasing as [3 is lowered): the 
dynamics is homogeneous, and all the spins have more 
or less the same relaxation time (with fluctuations). For 
a fixed connectivity equal to 6, on the contrary, we see 
that small values of t still keep a finite weight, while a 
second part of the curve, corresponding to large times, 
emerges as T decreases. This second part appears at 
temperatures for which aging dynamics sets in (i.e. at 
which C'(t w + t, t w ) becomes a function of both t and t w 
for all times), thus signaling the onset of a glassy regime. 
The cusp in the curve, around t* w 10 4 MC steps per 
spin, shows that a separation of time scales occurs. Such 
a cusp identifies "fast" and "slow" degrees of freedom, 
and persists for a large range of values of (3: the fraction 
of fast spins, J* P(t)cLt, is a slowly decreasing function 

of (3. For the case of connectivity 4, J Q P(t)cIt shows 
instead a sharp transition when the mean value of the 
relaxation times crosses t*. We show in figure (Q) the 
two shapes of P(r), for connectivity 4 and 6, and the 
evolution of J Q P(t)cLt with f3. 

If we now consider random hyper-graphs with fluctuat- 
ing connectivity, we observe a crossover between the two 
situations, as 7 (and therefore the mean connectivity C) 
is increased. Since the connectivity can vary from one 
site to anothe r, the global P(t) can be decomposed in a 
sum of P z (r) , distributions of times Ti restricted to the 
sites with connectivity z. Then, for 7 < 70, the P z (j) are 
smooth functions peaked around a mean value (evolving 
with (3), while, as 7 grows, the P z (j) becomes broader 
and broader, overlap with each other, and exhibits cusps 
Ipjfl . The crossover occurs around 70 (note that mean 



connectivity C = 4 corresponds to 7 < 70, and mean 
connectivity C = 6 to 7 > 70), thus indicating that the 
loop structure is responsible for the appearance of com- 
plicated, inhomogeneous dynamics. 

In order to understand the Glauber dynamics mecha- 
nisms on a microscopic level, we analyze in detail single 
samples of random hyper-graphs (with 7 > 70). Rather 
than averaging over disorder, we compare single runs and 
average over initial conditions. Each single run (i.e. ini- 
tial configuration) leads to a broad distribution of Tj. 
However, two cases may be distinguished. (I) if the con- 
nectivity can fluctuate from site to site, Ti does not have 
a strong dependence on the initial conditions: if we call 
and 7"^ the values of n for two independent runs, 
we see in the inset of figure (^) that the histogram of the 
ratio t^ 1 /t^' is sharp and close to one. Thus, the broad 
distributions of Tj lead to a broad distribution of (Ti), 
and the relaxations of the single site correlation func- 
tions Ci(t,t') depend strongly on the site z,(see Fig. |^). 
This shows that the position of the slow and fast degrees 
of freedom are encoded in the topology of the graph. (II) 
On the contrary, for random hyper-graphs with uniform 
connectivity, the histogram of figure (§) shows that the 
distribution of t\ Jt^ is broad (for a given site i, Ti can 
vary a lot from one run to the other). It follows that (t^) 
tends to a value independent of i in the limit of many 
runs, and therefore (si(t)si(t')) = (si(t)si(t')) . The slow 
or fast character of a spin depends on the initial condi- 
tions and is induced dynamically. Such a self induced 
frustration is probably more similar to what happens in 
real glasses. 

Surprisingly enough, both for random and constant 
connectivity hyper-graphs, (r^) does not depend on the 
nature of the couplings, either ferromagnetic or random. 
Once the loop structure of the graphs becomes irregular, 
frustration is totally induced by the dynamics through 
the initial conditions. For the sake of completeness, we 
have repeated the simulations on systems with Gaussian 
continuous couplings. All the discussed dynamical fea- 
tures are retained |2FJ . 

As a concluding remark, let us notice that, while the 
scope of this study was to show how mean field models 
could provide non-trivial local dynamical information, 
similar results are found in finite dimensional Edwards- 
Anderson spin glasses. In the latter case the topology 
of the lattice is fixed and the way of generating an ef- 
fective non trivial topology is by randomly adding an- 
tiferromagnetic bonds to the ordinary Ising model [fll"|| . 
For the Edwards- Anderson spin glasses with equally dis- 
tributed ferro and antiferro bonds, we have also simu- 
lated aging dynamics in two and three dimensions [^l| , 
with the scope of pushing further the st udy of ||. We 
indeed observe a similar shape for P(t). For ferromag- 
netic models on regular lattices instead, the well known 
coarsening relaxation process occurs: all spins are equiv- 
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alent and no rich structure of P{r) or long-lasting spatial 
heterogeneities can be found. This is also true for spin 
models on quasi-periodic lattices, and on random graphs 
(i.e. with K = 2 instead of K = 3): the dynamics shows 
a similar P(r), with heterogeneities, only for random in- 
teractions (Viana-Bray model), and not for ferromagnetic 
ones. 
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1. Typical examples of the distribution of times r be- 
tween two flips of a spin, P(t), for a hypergraph of N = 1000 
spins, with fixed connectivity; lines: connectivity 4, /3 = 4 
and /3 = 5; symbols: connectivity 6, f3 = 3; t w = 10 4 MC 
steps, T m ax = 10 6 MC steps. Inset: evolution of the fraction 
of fast spins, J P(r)dr with f3, for connectivity 4 and 6. 
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FIG. 2. For one given random hypergraph, local autocorre- 
lation function d(t w + 1, t w ) versus t for various sites i and a 
given t m . For some sites i d decreases very fast, while for oth- 
ers it evolves very slowly. The line is the 
Here N = 1000, 7 = 1, f3 = 3.5, t w = 10 4 MC steps,and 
the mean is over 300 runs. Inset: histograms of the ratios 
Tj^/Tj between two runs with different initial conditions 
for a random hypergraph with fixed connectivity (dotted line) 
and without (full line). 
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